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Abstract 

Background: The Veterans Health Administration (VHA) oversees the largest integrated healthcare system in the 
United States. The feasibility of a large-scale, nationwide, group-randomized implementation trial of VHA outpatient 
practices has not been reported. We describe the recruitment and enrollment of such a trial testing a clinician- 
directed, Internet-delivered intervention for improving the care of postmyocardial infarction patients with multiple 
comorbidities. 

Methods: With a recruitment goal of 200 eligible community-based outpatient clinics, parent VHA facilities 
(medical centers) were recruited because they oversee their affiliated clinics and the research conducted there. 
Eligible facilities had at least four VHA-owned and -operated primary care clinics, an affiliated Institutional Review 
Board (IRB), and no ongoing, potentially overlapping, quality-improvement study. Between December 2003 and 
December 2005, in two consecutive phases, we used initial and then intensified recruitment strategies. 

Results: Overall, 48 of 66 (73%) eligible facilities were recruited. Of the 219 clinics and 957 clinicians associated 
with the 48 facilities, 168 (78%) clinics and 401 (42%) clinicians participated. The median time from initial facility 
contact to clinic enrollment was 222 days, which decreased by over one-third from the first to the second 
recruitment phase (medians: 323 and 195 days, respectively; p < .001), when more structured recruitment with 
physician recruiters was implemented and a dedicated IRB manager was added to the coordinating center staff. 

Conclusions: Large group-randomized trials benefit from having dedicated physician investigators and IRB 
personnel involved in recruitment. A large-scale, nationally representative, group-randomized trial of community- 
based clinics is feasible within the VHA or a similar national healthcare system. 



Introduction 

Implementation research is the scientific study of meth- 
ods to promote the rapid uptake of research findings 
and, hence, improve the health of individuals and popu- 
lations [1]. Group-randomized trials (GRTs) are an 
increasingly important tool for implementation research. 
Typically, individuals {e.g., clinicians) are clustered 
within subunits (e.g., clinics) that may be further clus- 
tered within higher-level units (e.g., facilities or health 
systems). Accordingly, the unit of randomization and 
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the intervention target may be different (e.g., clinics and 
clinicians, respectively). Unlike the traditional rando- 
mized clinical trial (RCT), which focuses on efficacy, 
implementation research focuses on effectiveness [2,3]. 
The goal is to understand how efficacious interventions 
delivered in relatively homogenous populations can be 
deployed within the community to benefit the popula- 
tion at large. Thus, external validity (generalizability) of 
GRTs depends on the extent that participants at differ- 
ent levels of clustering represent the population of 
interest. 

Recruitment is important for traditional RCTs, pri- 
marily to achieve the needed power to detect significant 
differences in outcomes; for GRTs, recruitment is 
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important to ensure power and generalizablity. The 
Myocardial Infarction Plus Comorbidities (Mi-Plus) 
study was a nationwide GRT of Veterans Health Admin- 
istration (VHA) primary care clinicians who cared for 
ambulatory post-myocardial infarction (MI) patients, 
many of whom had multiple comorbidities. The 27- 
month clinician-directed, Internet-delivered intervention 
consisted of quarterly case-based interactive educational 
modules, one to three reviews per month of recently 
published studies of high clinical impact and relevance 
to the quality indicators, summaries and links to guide- 
lines applicable to the care of post-MI patients, and 
downloadable practice tools and patient educational 
materials [4]. The website was developed using service- 
oriented architecture and design principles refined in 
prior studies [5,6]. Iterative usability sessions were used 
to refine the content. Clinicians in control clinics were 
provided a link to an existing VHA Office of Quality 
and Performance website that contained links to a wide 
range of clinical guidelines for various medical condi- 
tions (http://www.healthquality.va.gov/). 

Similar to other multicenter implementation studies, 
the clinic was the unit of randomization [7]. Perfor- 
mance improvement was calculated as the change (pre- 
intervention period vs. postintervention period) in the 
proportion of patients receiving each clinical indicator 
within the clinic [8]. Individual clinicians were 
embedded within community-based outpatient clinics 
(clinics), which were embedded organizationally, though 
not necessarily colocated, within VHA parent facilities 
(medical centers). This design necessitated several 
sequential and, at times, simultaneous recruitment 
efforts targeting individual clinicians, clinics, and facil- 
ities. This report describes those recruitment activities 
as well as the recruitment times and participation rates 
at the facility, clinic, and clinician level. 

Methods 

The VHA is the largest integrated healthcare system in 
the United States, with 153 medical centers and over 
900 ambulatory care and community-based outpatient 
clinics providing care to an estimated 5.5 million indivi- 
duals in 2008 [9]. Each facility typically consists of an 
acute care component, on-site outpatient clinics physi- 
cally located at the facility, and off-site outpatient clinics 
distributed across the region served by the facility. Many 
facilities are also affiliated with an academic medical 
center and support research activities. Research within 
the facility must be formally approved by the facility's 
Institutional Review Board (IRB) and its Research and 
Development (R&D) committee. Any research con- 
ducted at a clinic is governed by the policies of its par- 
ent facility. 



The study was funded through the VHA Health Ser- 
vices Research and Development (HSR&D) office [IHD 
04-387] and by a parallel National Institutes of Health 
study [R01 HL70786-02] [10,11]. We conducted forma- 
tive work with a panel of expert physicians using nom- 
inal group techniques to choose from among 36 
potential quality indicators for complex ambulatory 
post-MI patients that would be both most feasible and 
most valid [8]. We also conducted focus groups and 
case-vignette surveys of clinicians, including VHA clini- 
cians, to develop the intervention. The Birmingham VA 
Medical Center, Birmingham, AL, served as the study's 
coordinating center. After approval from its IRB and 
R&D committees, the Birmingham facility and its six 
affiliated outpatient clinics were the first study enrollees 
in November 2003. 

A priori, we planned a sample size of 200 clinics to 
provide > 80% statistical power to detect a 5% difference 
in improvement between intervention and control 
clinics for all of the primary clinical indicators over a 
range of assumptions. Our initial recruitment plan 
allotted six months to recruit the 200 clinics using a 
strategic approach of recruiting parent facilities using 
high-yield targets (Le., personal contacts) and leveraging 
regional leadership support for our study. As only one- 
third of the requisite clinics were recruited after eight 
months, we re-evaluated the initial recruitment proce- 
dures (phase 1) and revised them to improve recruit- 
ment in phase 2. 

Phase 1 facility recruitment protocol 

In the first phase of recruitment (April 2004-November 
2004), potential facilities for recruitment were identi- 
fied using the 2003 VA Station Tracking (VAST) data- 
base. To be eligible, facilities had to have an affiliated 
IRB; four or more eligible clinics; and no ongoing, 
potentially overlapping, quality-improvement project. 
The four-clinic requirement was relaxed towards the 
end of recruitment. Clinics were eligible if they were 
noncontract (owned and operated by VHA), delivered 
primary care, used the VHA's electronic health record 
(EHR) system, and provided Internet access to all clini- 
cians. For eligible facilities, we sought a physician will- 
ing to serve as a local principal investigator (PI). To 
identify funded investigators and other potential con- 
tacts within each facility who could serve as a local PI, 
the VHA R&D and HSR&D websites were reviewed. 
The subsequent list was reviewed by our study investi- 
gators to identify high-yield targets (i.e., personal con- 
tacts) to initiate facility recruitment. The project 
coordinator (a registered nurse) called and emailed 
these high-yield targets on behalf of the study 
investigators. 



Funkhouser et al. Implementation Science 201 1, 6:105 Page 3 of 1 1 

http://www.implementationscience.eom/content/6/1/105 



Phase 2 facility recruitment protocol 

In the second phase of facility recruitment (December 
2004-July 2005), "cold" contacts were recruited via 
emails and telephone by targeting facilities with the 



largest number of associated clinics. To increase recruit- 
ment and its efficiency, we developed and implemented 
a standardized recruitment protocol (Figure 1). During 
this second phase, we added two physician investigators 



Call (until reach one): 

Chief of staff for primary 1 care 

Head of research 

Chiefs of medicine, cardiology 

IRB chair 
Goal: Identify local PI 




Send email with recruitment materials* 



Send email with recruitment materials* 



7 Days 



Follow-up telephone call 
and email 






No 




F 


Continue telephone 


recruiting 



^Abstract, press release, recruitment letter 

Figure 1 Facility recruitment scheme (phase 2): the VA Mi-Plus study. 
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to assume primary responsibility for recruitment of local 
Pis and leverage physician-to-physician communications 
[12]. To facilitate the IRB approval process at participat- 
ing facilities, a research assistant was also hired to speci- 
fically oversee all IRB protocols and to prepare a 
standard IRB packet for each facility. In phase 1, 
research staff assisted each facility with IRB preparation 
but did not prepare a standard IRB packet. The 14 facil- 
ities for which a local PI could not be found or for 
which IRB approval was never obtained were classified 
as "declined" facilities (Figure 2). 

The physician investigators followed the recruitment 
protocol shown in Figure 1. Contact information 
(name, position, telephone number, and email) for 
potential physician Pis was obtained from the VAST 
database and facility websites. These initial emails con- 
tained recruitment materials (the study abstract, a 
press release, and a recruitment letter), outlined the 
need for a local PI, and described the general 



1 08 Facilities in VH A station tracking 
database 



expectations of this position. During the telephone 
call, questions were answered, interest was ascertained, 
and if the individual declined to participate, they were 
asked to refer others who might be interested. This 
process was continued until a local PI was identified 
or all leads were exhausted, including contacting the 
chief of medicine, IRB chair, and chief of staff. To 
cover costs of participation, the facility received a site 
distribution of $2,500. 

Facilities were recruited until we achieved our goal of 
200 eligible clinics. While facility recruitment continued 
in a rolling fashion, we simultaneously recruited clinics 
and clinicians of enrolled facilities to participate in the 
intervention study. Clinicians at each facility's associated 
clinics were not recruited or provided study materials 
until a local PI was identified, all IRB requirements were 
met, and a list of all eligible clinicians and their email 
addresses were obtained. The date these materials were 
approved and posted was the facility's launch date. 



10 Facilities identified during recruitment 
(0 extension, 2 new) 



1 18 Total facilities 



52 Facilities were ineligible 



42 Had less than 4 community- based 
outpatient clinics 



7 Had no IRB 



3 Already participating in a 
quality- improvement study 
relevant to Ml patients 



66 Facilities were eligible 



14 No local PI found 



52 Agreed to 
participate 



4 Withdrew prior to 

completing the 
enrollment process 



48 Participated 



Figure 2 Facility participation: the VA Ml-Plus study. 
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Clinic and clinician recruitment protocol 

A clinic was enrolled and randomized when the first eligi- 
ble clinician (a physician, physician assistant, or nurse 
practitioner) at that clinic logged on to the study website. 
All clinicians at a clinic were randomized to the same arm, 
but only clinicians who logged on were enrolled. Clinicians 
were recruited continuously throughout the two-year 
intervention period. Immediately following the facility 
launch, clinicians were sent an email, a postal letter, and 
study flier that described the study and how to log on to 
the study webpage. Subsequent weekly email and fax 
reminders were sent to clinicians who had not yet logged 
on. Approximately four to six weeks after the facility 
launch date, one of the study physicians sent a more per- 
sonalized email to each clinician at clinics not yet enrolled 
(this involved seven clinics over the course of the study). If 
unsuccessful, telephone contact was attempted with each 
clinician at clinics not yet enrolled. Telephone attempts 
were discontinued if a clinician was reached or three 
attempts were made. The date and type of contact attempt 
was tracked in an Excel (Microsoft Corporation, Red- 
mond, WA, USA) spreadsheet; however, more recent con- 
tact attempts were overwritten on earlier attempts. The 
primary goal of these attempts was to increase clinic 
enrollment and not clinician enrollment (i.e., if any clini- 
cian at a given clinic logged on, the clinic was considered 
to be enrolled). Proactive emails were sent notifying all 
clinicians (enrolled or nonenrolled) of new updates and 
materials. Such reminders have been demonstrated to 
increase participation in Internet-delivered clinician inter- 
ventions [13]. Lastly, a monthly recruitment report was 
emailed to the local PI at the associated parent facility. 
The report contained the name of each clinician and his/ 
her enrollment status. Local Pis were encouraged to infor- 
mally facilitate recruitment where feasible by encouraging 
their peers to log on. All enrolled intervention and control 
clinicians could obtain continuing education credits for 
reviewing eligible educational materials on the website. No 
other incentives were provided owing to VHA policy. 

Statistical analysis 

Differences in facility participation rates were assessed 
according to the presence of a formally funded existing 
VHA HSR&D program at the time of recruitment 
(defined as a Center of Excellence, Research Enhance- 
ment Award Program, or a Targeted Research Enhance- 
ment Program), rural-urban locale, [14] geographic 
region of the United States, and facility size in terms of 
number of affiliated clinics. Differences in clinic partici- 
pation rates among participating facilities were similarly 
assessed, with clinic size classified according to the 
number of affiliated clinicians. The analyses were 
repeated among participating facilities to assess differ- 
ences by recruitment phase. 



We defined four time intervals to represent the differ- 
ent aspects of the total recruitment time for a clinic: (1) 
initial facility contact to recruitment of a local PI, (2) 
recruitment of a local PI to approval by both the R&D 
committee and the IRB, (3) IRB or R&D approval (which- 
ever the facility required last) to launch, and 4) launch to 
first clinic enrollment. Interval 2 represents an estimate 
of facility approval time. Because all time intervals were 
skewed, the median was used as a measure of central ten- 
dency. Kruskal-Wallis tests were used to assess differ- 
ences of time variables across categorical variables, and 
Spearman rank correlations were used to measure asso- 
ciations of selected characteristics, specifically, measures 
of facility and clinic size, with time intervals. 

Results 

Facility participation 

Of 118 facilities identified as potentially eligible (Figure 
2), 66 were confirmed eligible. Of the 66, 12 had a for- 
mally established VA HSR&D program and 59 were in 
an urban area (Table 1). The largest proportion (n = 23, 
35%) was in the midwest and the lowest in the west (n 
= 12, 18%). The median number of affiliated clinics was 
five (interquartile range of four to six). Forty-eight facil- 
ities (73%) located across the continental United States 
(Figure 3) participated. Facility participation rates were 
higher among facilities with a formally established VA 
HSR&D program, those located in an urban area, those 
not in the west, and those that were smaller (Table 1). 

For the 18 facilities that did not participate, a willing PI 
could not be identified in 14 (Figure 2), with two facilities 
citing serious staffing problem/staffing turnover as reasons. 
Willing Pis were found at another 4 of the 18 facilities but 
their research offices declined for two of them, and another 
two never completed the IRB approval process. 

Clinic participation 

There were 219 clinics affiliated with the 48 participat- 
ing facilities, of which 168 (77%) participated (Table 2). 
As with facilities, most clinics were located in urban 
areas, with relatively few in the west. The median clinic 
size (number of clinicians) was 3 (range 1 to 15). Larger 
clinics and clinics located in urban areas were more 
likely to participate than their counterparts; clinic parti- 
cipation did not differ by US region. 

Clinician participation 

There were 957 clinicians affiliated with the 219 clinics, 
of whom 401 (42%) participated (Table 2). In contrast 
to clinic participation rates, clinician participation rates 
did not differ by rural-urban locale or clinic size. As 
with clinics, clinician participation rates did not differ 
by geographic region within the continental United 
States. 
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Table 1 Distribution of eligible facilities according to participation: the VA Ml-Plus study 
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4 
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*p for trend test; **p for Wilcoxon rank sum test. 
VHA = Veterans Health Administration. 



Facility recruitment time 

Excluding the coordinating center, the facilities were 
recruited over a 15-month period. Between April 2004 
and November 2004, 16 facilities (25% of 65 eligible) 
were recruited, and between December 2004 and July 



2005, 31 facilities were recruited (63% of 49 remaining 
eligible) (Figure 4). The median time from initial facility 
contact to clinic enrollment was 222 days. This interval 
decreased by over a third, from a median of 323 to 195 
days (p < .001), from the first to second recruitment 
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Table 2 Distribution of community-based outpatient 
clinics and associated clinicians among the 48 
participating facilities: the VA Ml-Plus study 

Clinics Clinicians 



Total Participating Total Participating 
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63 
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18 
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Puerto Rico & Virgin 
Islands 
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39 
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107 
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> 4 


88 


79 


89.8% 
p < .001 


691 


277 


40.1% 
p = .2 



a Does not include the six clinics and 29 clinicians from Puerto Rico & the 
Virgin Islands. 



phase (Figure 5). This was largely due to a decrease in 
facility approval time (255 to 94 days; p < .001), which 
remained the largest component of recruitment time. In 
all but three facilities, approval by the R&D committee 
was required before submission for IRB approval. This 
initial approval constituted 80% (93 of 116 days) of over- 
all median facility approval time, a percent similar for 
both phases of the study. Initial contact time (i.e., time 
to identification of local PI) did not differ by facility size 
overall (r = .01) or in either phase. Facility approval 
time was associated with facility size in the first phase (r 
= .68) but not the second phase (r = -.01). As expected, 
clinic size was inversely (r = -.51) associated with time 
from facility launch date to clinic enrollment (participa- 
tion); namely, clinics with more clinicians had a clinician 
log on sooner than did clinics with fewer clinicians, and 
this association was present in both recruitment phases 
of the study. 

Clinic/clinician recruitment time 

Over half (n = 90; 53%) of the clinics enrolled within 
one week of facility launch and most (n = 146; 87%) 
enrolled within four weeks. Six weeks after facility 
launch, only 7% of clinics had not enrolled. This pattern 
was the same for both recruitment phases. The longest 
time period to enroll a clinic was 10 weeks (n = 3 



clinics). Regarding time to last new clinician logging on 
within a clinic, 25% of clinics had the last new clinician 
participate by four weeks, 50% by 7.5 weeks, and 75% by 
28 weeks. One clinic had a clinician who first logged on 
80 weeks after initial invitation. Although we did not 
formally gather information on clinician refusal, which 
was passive, qualitatively, most clinicians who did not 
enroll and were reached by telephone cited lack of time 
and interest as reason for not participating. 

Discussion 

The possibility of obtaining a large, nationally represen- 
tative sample of primary care clinicians (physicians, phy- 
sician's assistants, and nurse practitioners) makes the 
VHA health system an enticing setting to conduct 
implementation and outcomes research. With careful 
planning, a systematic yet flexible approach, and a mul- 
tidisciplinary staff, it is possible to recruit a nationwide 
sample of primary care clinicians employed in the 
VHA's community-based outpatient clinics. Over 
approximately two years, we were able to recruit 401 
clinicians representing 168 clinics and 48 facilities in 26 
states and Puerto Rico and the Virgin Islands. These 
groups accounted for 73% of all eligible facilities, over 
75% of their associated clinics, and 42% of their 
clinicians. 

Most GRTs do not report a response rate as they have a 
target number of "groups" or practices to recruit for the 
purposes of statistical power [15] and do not identify, or at 
least report, a sampling denominator. Our facility and 
clinic response rates were much higher than the 27% of 
nursing homes in a GRT study of osteoporosis fracture 
prevention [16] or the 33% of practices in a managed-care 
organization's study to increase chlamydia screening [5]. 
Our response rate is similar to non-GRT studies where 
the purpose was to obtain a population-based nationwide 
sample. For example, the National Institutes of Health- 
funded Coronary Artery Risk Development in Young 
Adults (CARDIA) study has been following an initial 
cohort of 5,115 community-dwelling healthy young adults 
first recruited in 1985 for nearly 25 years. The initial 1985 
recruitment for CARDIA resulted in a 55% response rate 
[17]. CARDIA has significantly contributed to our scienti- 
fic knoweldge, having resulted in over 400 peer- reviewed 
publications. More recently, the National Institues of 
Health-funded Cancer Outcomes Research Consortium 
(CanCORS) was established in 2001 to obtain a represen- 
tative, population-based sample to study the processes and 
outcomes of patients with newly diagnosed lung or color- 
ectal cancer [18]. The approximately 10,000 cancer 
patients recruited with a population-based approach 
represent about 50% of the underlying target population. 
As the recruitment methodologies of GRTs become more 
refined, their findings will be highly generalizable. 
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Figure 4 Cumulative enrollment by month: the VA Mi-Plus study. 



Recruiting for GRTs and for RCTs can be viewed 
under similar theoretical perspectives, including Choo's 
general model of information use identifying major ele- 
ments that influence information-seeking behavior [19] 
and the work of Christensen and Armstrong involving 
diffusion of innovation [20], which includes "disruptive" 
effects. In the VA Mi-Plus study, recruitment involved 
two groups of clinicians: (1) physicians to identify a 
local PI and (2) clinicians to log on and participate in 
the intervention. These clinician groups may have differ- 
ent elements that influence their participation. Local Pis 
had to complete necessary IRB training and submit 
applications through R&D and IRB committees for 
study approval at their facility. Even with the parent site 
(Birmingham) preparing necessary packages in the sec- 
ond phase, obtaining these approvals could be quite 
time consuming. There was no direct compensation to 
these individuals. Reasons to participate, as cited by 
another GRT [21], may include the desire to improve 
their clinical practice or an interest in contributing to 
medical knowledge in general, but these benefits must 
exceed any perceived disruptive effects. In comparison, 



at the clinic level, a clinician simply had to log on to 
enroll and thus be classified as participating. 

In comparing phase 1 and phase 2 recruiting, we 
found, as have others [21,22], that physician-to-physi- 
cian recruiting gave a much greater yield and that prior 
personal contacts did not have a substantial effect. We 
also learned that recruitment strategies needed to 
change over time in order to achieve recruitment tar- 
gets. Similarly, Ellis et al [21] used 10 different nonran- 
domized strategies over 11 months to recruit sufficient 
practices in the GLAD HEART study, a total of 61 prac- 
tices, all within one US state. In a review of recruitment 
rates and strategies across studies conducted in one 
medical center, Johnston et al. [23] found considerable 
variation in recruitment rates despite similar strategies 
and staffing. Number of recruited practices ranged from 
30 to 137; most required over nine months to recruit 
and most had not planned for the time needed. They 
found personal connections helpful and have suggested 
that these personal connections can be developed during 
the recruiting process. We also found that buy-in from 
participants (the use of local Pis to champion the study) 
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Figure 5 Median number of days for component intervals from initial facility contact to first clinic enrollment: the VA Ml-Plus study. 



and a flexible recruitment strategy enhanced recruit- 
ment, findings consistent with those of Johnston et al. 
[23] 

Minimization of possible disruptive effects for the 
clinician may have facilitated recruitment in our study. 
First, VHA's use of EHRs made it possible to extract 
patient records without interfering with office flow. 
Also, randomization and analysis was at the clinic level, 
thus low-performing individual clinicians were not at 
risk of being identified. Similarly, the use of EHRs and 
clinics as the randomization unit enabled the recruit- 
ment of 20 practices in 14 states for a multimethod 
GRT [24]. The Ornstein study relied on academic detail- 
ing and site visits, components that may be disruptive 
from the theoretical perspective and expensive or 
impractical for a nationwide study. Interestingly, the 
parallel Mi-Plus study involving primary care clinicians 
in Alabama and Mississippi [25,10,8] had a much lower 
participation rate (13%) for clinicians [25], perhaps 
because these clinicians lacked EHRs and viewed manual 
chart abstraction as disruptive to their practices. 

Between the first and second phases of our recruit- 
ment, the amount of time required to obtain facility 
approval of the study protocol decreased from a median 



of 255 days to 94 days. This 63% reduction was primar- 
ily attributed to the addition of an experienced IRB staff 
member at the Study Coordinating Center that allowed 
for the implementation of a more systematic and struc- 
tured approach to IRB management. The complexity 
and sheer volume of work needed to coordinate IRB 
approval for 48 participating facilities cannot be over- 
stated. The majority of facilities required R&D approval 
prior to IRB submission, and obtaining R&D approval 
constituted the bulk of the facility approval time, with 
IRB approval requiring only an additional two to four 
weeks. This may be misleading in that many R&D com- 
mittees wanted "the essence of the IRB packet" to 
review, thus, an IRB specialist is invaluable in facilitating 
R&D approval as well. 

Establishment of the recently implemented central IRB 
in the VHA (an IRB approved by a central office to 
cover all participating facilities in a multisite study) 
should enhance the efficiency, cost, and attractiveness of 
conducting nationwide GRTs within the VHA. Use of 
single-study IRB cooperative agreements in the (beta)- 
Carotene and Retinol Efficacy Trial (CARET) in a uni- 
versity setting reduced the average time to complete IRB 
approval from over six months to one month for each of 



Funkhouser et al. Implementation Science 201 1, 6:105 
http://wwwjmplementationscience.eom/content/6/1/105 



Page 10 of 11 



many substudies [26]. Even with a central IRB, we 
anticipate, as have others [27-29], that a dedicated 
research assistant or IRB specialist is advised in the 
planning of any large GRT within or external to the 
VHA. In 2005, with an established protocol and experi- 
enced staff, it took approximately six months from 
initial contact at a facility to enroll an associated clinic; 
half of this time (three months) was for facility approval, 
which perhaps can be reduced to one month with the 
central IRB recently implemented by the VHA. One 
challenge that will remain, even with a central IRB, is 
getting Pis to do requisite training in research practices 
(e.g., good clinical practices, privacy, and security train- 
ing) needed for IRB approval. This required substantial 
effort from our study staff, primarily that of the IRB spe- 
cialist. In an era of ever-increasing regulatory oversight, 
we believe that this will persist as a substantial task that 
should be planned for when designing studies and bud- 
geting personnel. A database of and for VHA research- 
ers to register and complete the approval and training 
necessary to do VHA research should facilitate the 
recruitment process. 

Our conclusions regarding the importance of a func- 
tional, truly interdependent relationship between the 
study PI and the clinical research coordinator echo 
those of other teams [30]. The success of our study 
would not have been possible without a close collabora- 
tion between these two members of the research team. 
Evaluating the value-added contribution of such a posi- 
tion should be an important future consideration. 

Our experience suggests that using a recruitment 
approach that seems counterintuitive might be war- 
ranted. Our initial efforts to recruit local Pis focused on 
high-yield targets (i.e., personal contacts), largely due to 
initial anxiety on the part of the recruitment team of 
cold calling. While recruiting based on familiarity might 
have made us feel better, the cold peer-to-peer calling 
successfully recruited many local Pis and proved less 
difficult and more efficient than anticipated. We might 
have saved time and improved study efficiency by 
expending more energy on cold calling local Pis early 
and getting the recruitment process started and saving 
the "easy" recruits for later. Anecdotally, cold calling 
individual clinicians to log on was not nearly as success- 
ful a recruitment tool as cold calling for local Pis. This 
observation may be a result of being able to offer the 
facility of local Pis a site distribution of funds ($2,500) 
to cover costs of participating, while we could not offer 
clinicians any similar distribution of funds for participa- 
tion in the study owing to VHA policy. 

Conclusions 

We found that having dedicated research team mem- 
bers, physician investigators, and an IRB specialist 



actively involved in the recruitment process and using a 
standardized recruitment protocol greatly increased the 
ability and efficiency of facility recruitment. These spe- 
cialized personnel, however, appeared to have very little 
effect on recruiting clinics and clinicians. We believe 
that our study demonstrates the ability to do implemen- 
tation research with a level of generalizability compar- 
able to that of major epidemiologic studies. As group- 
randomized implementation trials become more com- 
mon, large healthcare systems, such as the VHA, will 
provide us with the opportunity to refine our methods 
and become key "laboratories" for the development of 
implementation science. 
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